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Preface 



I n December 1994, a roundtable was convened to examine the topic of core 
indicators of effectiveness in community colleges. The members of the 
roundtable chose several questions to guide the discussion: What is effec- 
tiveness and how is it different from concepts like efficiency, student outcomes, 
and performance assessment? Are community colleges different from other 
institutions when it comes to measuring effectiveness? Is effectiveness based on 
the mission of a college, or is it inherent in what stakeholders need and want 
from colleges? What should be measured and why? What is a “core indicator,” 
and how does it differ from other measures? 

The discussion resulting from these questions touched on a number of ideas 
about how to define and measure effectiveness. The AACC special report 
Community Colleges: Core Indicators of Effectiveness summarized the discussion 
and synthesized different indicators for measuring effectiveness into a compre- 
hensive model. This report was distributed to community colleges throughout 
the nation and became a benchmark for institutions beginning or expanding 
their efforts to define and measure effectiveness. By the mid-1990s, many col- 
leges had adopted effectiveness models incorporating the core indicators. 

The context in which our colleges operate has changed, however. The mil- 
lennium is ushering in new forces, and stakeholders are looking at institutions 
with different eyes. Policymakers are interested in “high performance” and 
what institutions can and must do to get better results. For example, what must 
colleges do to retain more students, improve transfer rates, and see more stu- 
dents through to degree completion? Employers are interested in response time 
and customization. How quickly can institutions develop programs that meet 
workforce needs? And students want to be treated like customers. They want a 
relationship with their college that offers convenience, value, and quality, just 
as they would expect from a bank or other service provider. 

In view of these developments, the objective of this second edition of Core 
Indicators of Effectiveness is to update the first report by examining changing 
conditions and new thinking related to effectiveness. Chapters 1 and 5 address 
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the changing context in which our colleges operate and new directions in 
assessment. These sections wrap around the main body of the report, chapters 
2 through 4, which provide a working definition of effectiveness, a model 
depicting its components, and a list of the core indicators organized according 
to mission arena. Chapter 4 includes detailed technical descriptions, along with 
data sources and additional measures, for each indicator. The report closes with 
a plan of action for community colleges in chapter 6. 

Community colleges are at the center of changes that will reshape their com- 
munities. The colleges have already made progress in designing and building 
systems for assessing effectiveness, but much remains to be done in measuring 
performance with stakeholders. Only a minority of colleges can produce 
credible data documenting student and institutional performance, and colleges 
continue to be vulnerable to policymakers’ questions about performance on 
traditional measures of student success. We hope this second edition of Core 
Indicators of Effectiveness will be as useful as the first in helping practitioners 
tackle important issues in effectiveness. Beyond this, we hope that it will shed 
light on emerging themes in effectiveness, while helping colleges answer tough 
questions from stakeholders in a customer-focused, externally driven market. 



Richard Alfred 
University of Michigan 
Ann Arbor 
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The Changing Context 
for Effectiveness 



A s community colleges approach the new millennium, transformational 
forces are compelling them to raise fundamental questions, to make 
hard choices, and to implement necessary changes so they can improve 
performance. Colleges are responding by involving stakeholders more deliber- 
ately in decisions about resources. Arms-length transactions between institu- 
tions and their constituencies are being replaced by aggressive alliances and 
outreach in which intangibles such as service, innovation, and flexibility become 
essential to success. New indicators of performance are drawing interest and 
becoming part of the effectiveness equation. 

Among the transformational forces at work inside and outside our colleges, 
three are so pervasive that to ignore them is to invite trouble: changing 
markets, technology, and the drive for performance and accountability. 



Changing Markets 

Students with changing needs and expectations. A powerful new wave of 
students with intensifying needs and expectations is hitting college campuses. 
Students and clients outside of college walls, such as employers, are becoming 
more and more critical of the quality of service they receive, and they expect 
something to be done about it. Students now say, “I want terrific service. 1 want 
convenience. I want quality. Give me classes 24 hours a day, and give me 
close-up parking if possible.” Students do not want to pay for anything 
they are not using (Levine 1998). They want convenience, responsiveness, and 
flexibility, and they want it now — or they will go somewhere else 
to find it. 

Two distinct attributes of this intensifying demand have important implica- 
tions for effectiveness. A college’s efforts to measure effectiveness must include 
not only its ability to meet current needs but also its ability to deliver what cus- 
tomers expect if they are to be pleased with the quality of service. Students have 
an expanding array of options for postsecondary education and quickly become 
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familiar with programs and services they once considered novel. The more 
experience students have with a program or service, the more discerning they 
become about their own needs and the variety of ways available for meeting 
those needs (Alfred 1999). As they progress along a scale of experience and 
exposure to what institutions have to offer and become familiar with other 
options, their needs begin to shift and their expectations to rise. Raised expec- 
tations are difficult to diminish. They drive up the standards against which stu- 
dents express satisfaction and create a new level of service expectation. 

New competitors. Until recendy, competition involved predictable relationships 
among known competitors in a stable market. Today, however, the environ- 
ment is one of unpredictable and turbulent change. New competitors are 
reshaping the postsecondary education market. The University of Phoenix, the 
fastest-growing educational institution in the world, serves 70,000 students on 
100 campuses in 32 states and offers an online campus with complete degrees. 
Phoenix is not alone. A growing number of for-profit higher education compa- 
nies (DeVry, Inc.; ITT Educational Services, Inc.; Education Management 
Corporation; Computer Learning Centers) are setting up shop in storefronts 
near campuses. 

Other players include the cable industry giants, the corporate universities 
(the fastest-growing sector in higher education), the course software develop- 
ers, and the burgeoning array of new public and private partnerships. There are 
also the virtual universities, electronic campuses, and electronic community 
colleges, all without geographic boundaries (McClenney 1998). 

In a competitive market, success comes to organizations that distinguish 
themselves in the eyes of customers (Alfred 1999). The University of Phoenix, 
for example, stands out because it has developed systems to quickly develop 
and deliver new programs to clients. Telephone and cable companies make 
education convenient through distance delivery into homes, community cen- 
ters, shopping malls, and just about anywhere people congregate. Corporate 
training programs provide unique ways of connecting education and work. For 
example, Motorola and General Electric have developed customized training 
programs that give workers skill sets that can be transported to many different 
businesses. And we should not overlook the reputation for customer service 
that some proprietary institutions have established by totally reinventing 
processes such as student intake, placement, and financial aid delivery. 
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Technology 

External forces are driving heightened expectations for technology on commu- 
nity college campuses. One factor is the increase in computer ownership 
among American families. Today more than 50 percent of U.S. homes have at 
least one computer (McClenney 1998). In 1997, for the first time, computers 
outsold televisions. Twenty-three percent of the population of the United States 
and Canada uses the Internet. By the beginning of 1999, an estimated 80 
percent of U.S. public schools will be online. 

In the United States, corporate spending on information technology 
surpassed expenditures on manufacturing technology several years ago. 
Computer technology is ubiquitous, cutting across income categories as well as 
educational and occupational lines. Technicians, farmers, small business own- 
ers, office workers, and degreed professionals are all likely to use computers in 
their daily work. 

Demography also fosters expectations about the impact of technology on 
community colleges. Demographic data point to a rising demand for postsec- 
ondary education that has not been matched by gains in campus capacity. 
Distance education fuels rising expectations among state officials that technolo- 
gy can resolve some pressing capacity problems for less money than the costs 
associated with expanding campus facilities, building new campuses, or hiring 
more faculty (Green 1996). Policymakers are now concluding that traditional 
place-based, credit-for-contact educational models are too expensive to meet the 
rising demand for educational services (McClenney 1998). Technology will pro- 
foundly affect virtually every aspect of education: what students learn, how they 
learn, and when and where they learn. 



Performance and Accountability 

Pressures for performance documentation are intensifying for almost every 
constituency served by community colleges. The passage of Student Right-to- 
Know legislation in the Higher Education Act and a congressionally created 
blue-ribbon panel on college costs are two of the more recent manifestations of 
a trend that has been accelerating since the 1970s and is apt to intensify in the 
next century. The inescapable conclusion is that policymakers and the public 
are through with signing blank checks for higher education. Colleges are going 
to be expected to perform, to document their performance, and to be account- 
able for producing return on taxpayer and student investment (McClenney 
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1998). They are going to see this dynamic reflected in performance indicators, 
performance funding, performance contracting, and performance pay 

Accountability works as both an incentive and an obstacle for our colleges. 
As an incentive, it pushes colleges to develop performance models and indica- 
tors at a faster pace than would result from natural organizational processes. As 
an obstacle, it limits the terrain for interpretation of institutional performance 
to subjective criteria based on limited experience. All too often, these substi- 
tutes for direct experience drive accountability expectations, although they 
may have little or nothing to do with the mission of the institution or its 
performance. 

A decade of rhetoric and effort has produced minimal change in perfor- 
mance assessment. Only a small number of colleges can produce credible data 
documenting student academic performance and progress. Our colleges 
continue to be vulnerable to policymakers’ questions about performance on 
traditional measures of student success (graduation rate, time to degree, reten- 
tion, transfer rate). And the colleges cannot mount an effective campaign to 
persuade policymakers to pursue alternative measures, because data is not 
available to make the case. 



Implications for Effectiveness 

One major conclusion that can be drawn from this description of transforma- 
tional forces is that effectiveness involves a push-pull effect for community col- 
leges. Most institutions are focusing their effectiveness packages on mission 
achievement and student performance and progress. However, as colleges rely 
more on information and support from external stakeholders, they will need to 
supplement traditional academic indicators of performance with other indica- 
tors. Stakeholders will not readily commit resources to institutions that do not 
meet their expectations, or appear not to because they do not provide evidence 
to the contrary (Alfred 1998). Employers will not enter into contractual rela- 
tionships with colleges that do not meet their quality, cost, and service require- 
ments. Baccalaureate institutions will be hard-pressed to admit students from 
community colleges that do not provide a foundation in basic skills. 
Government agencies will not provide full resources to colleges that do not 
document a return on investment by publishing information about costs and 
degree and program completion. Community organizations will be reluctant to 
commit resources to institutions that do not get involved with important 
community development issues. 
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To meet this challenge, community colleges will need to ensure that their 
effectiveness systems are flexible and dynamic. These systems will change 
shape according to the demands of the market. Successful institutions will 
move beyond conventional approaches to effectiveness in which caution is 
valued more than action. Successful institutions will seek to create a fluid 
organization and to document performance using a blend of traditional and 
nontraditional indicators reflecting the expectations of multiple stakeholders. 
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What Is Effectiveness? 



ffectiveness is difficult to define and therefore eludes straightforward 



measurement. Like community colleges themselves, definitions of 



effectiveness change with the times. As argued in chapter 1, this is part- 
ly because external audiences are far more important than they were just a few 
years ago. Like professionals in healthcare and many other fields, insiders in 
higher education can no longer claim a monopoly in defining quality. Other 
change drivers include new fiscal and political conditions. Foremost among 
them are resource constraints due to taxpayer resistance, and new calls for 
accountability — both of which have resulted in demands for heightened 
productivity. 

Despite these changes, the heart of any definition of institutional effective- 
ness remains the ability of an institution to match its performance to its estab- 
lished purposes as stated in its mission (Ewell 1992). This implies that the 
institutions outcomes are the ultimate measure of whether or not it can be 
deemed effective. It also implies that the effective institution can conclusively 
document the outcomes it produces in support of its mission. Surrounding this 
definition, however, are two important caveats. 

First, both purposes and results must be consistent with a growing variety 
of stakeholder needs. No community college, regardless of the quality of its 
graduates, can be considered effective if its results are incongruent with its 
clients’ needs and expectations. As a consequence, specific judgments of col- 
lege effectiveness cannot be uniform. Instead, they will always vary according 
to the needs and values of a college’s stakeholders (Alfred and Kreider 1991). 
An institution’s willingness and capacity to determine and quickly respond to 
stakeholder needs is also an integral part of effectiveness. 

Second, results must be produced efficiendy within the constraints of avail- 
able resources. Under current financial and political conditions, no community 
college will be able to meet stakeholder needs without increasing costs — even if 
it wanted to hold the line on costs. Efficiency and effectiveness are different con- 
cepts, and community college leaders need to work hard to ensure that 
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simple efficiency measures are not substituted for real indicators of effectiveness. 
The wise use and responsible stewardship of available resources in the pursuit 
of mission-related goals must become an integral part of today’s definition of 
effectiveness. 

These key ingredients of institutional effectiveness — and their interrelation- 
ships — are shown in figure 2.1. A college or university is effective when its 
results match its mission, but both mission and results must also match stake- 
holder needs. In addition, these relationships must be maintained within the 
limits of costs and available resources. 



Figure 2.1 

Core Concepts in Institutional Effectiveness 
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Modeling and Measuring 
Effectiveness 



T he following definitions and guidelines provide the context for a 
discussion of core indicators. 

Definitions and Measurement Guidelines 

• Outcomes serve as benchmarks for the college mission by comparing 
results with purpose, within the limits of costs and available resources. The 
effective community college creates successful outcomes for multiple 
stakeholders, internal and external. 

• When outcomes are compared with an institutions mission and goals, the 
result is a composite picture of an institutions effectiveness. 

• An indicator that helps create this composite picture describes a condition 
or result that the community college can produce regularly report publicly, 
and use systematically in decision making. 

• A core indicator is a measure that describes a critical, widely recognized 
outcome of mission — one that is clearly responsive to key constituent 
groups and is produced regularly. 

The measurement of core indicators should reflect and support sound 
assessment practice. The following guidelines, framed as questions, make 
it more likely that the measurement of core indicators will conform to 
accepted practice and that specific measures will become useful tools for 
determining effectiveness. 

• Is the indicator supported by a comprehensive information system? 

• Is the indicator part of a commitment to track important data over time, 
ensuring that documenting and improving effectiveness is a developmen- 
tal process? 

• Is there a standard of comparison or a benchmark against which progress 
can be measured? 
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• Is the ongoing reliability of the indicator regularly assessed? Are changes 
made when needed? 

• Is the indicator credible to college personnel who are in a position to 
change institutional behavior? 

• Can the indicator be readily understood by external decision makers? Is it 
credible to them? 

• Does the indicator provide information that will help the institution to 
improve? Does it encourage the institution to value the right things? 

• Does the indicator reflect the perspectives and concerns of multiple 
constituencies? 

• Can data be obtained for the indicator at a reasonable cost? 



Modeling the Core Indicators 

A model is an abstraction of reality that can be used to improve our under- 
standing of concepts such as effectiveness. If effectiveness is defined as the 
capacity of a college to match its results with its mission and the needs of its 
stakeholders within the limits of costs and available resources, what would an 
effectiveness model look like? We know that community colleges have many 
stakeholders, but are they equally important? We also know that stakeholders 
view colleges differently depending on what is important to them — they have 
different performance expectations stemming from different needs. Finally, we 
know that colleges cannot measure everything — efficiency is important — and 
that means they must use a smaller number of indicators carried out with a 
limited number of stakeholders. 

These observations can be drawn together and represented in a comprehen- 
sive effectiveness model as illustrated in figure 3.1, which identifies different 
stakeholder groups — internal and external — and presents core indicators 
representative of different expectations. To enhance the efficiency of the model 
and to make it organizationally relevant to community colleges, the core indica- 
tors are clustered into the mission arenas commonly pursued by community 
colleges. For example, the indicator Degree Completion Rates is included in the 
Student Progress arena. The core indicators are not mutually exclusive and can 
be shared by internal and external stakeholders. Their importance will vary 
among stakeholders. 
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Figure 3.1 

Comprehensive Effectiveness Model 
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The core indicators are not random, isolated measures. As the model shows, 
they connect the mission of a college and the results it produces with the needs 
of its stakeholders. Practitioners are encouraged to use the model as a basis 
upon which to review existing effectiveness plans or to build new plans. 



Mission Arenas and Core Indicators 

Following are descriptions of 14 core indicators, grouped into six mission are- 
nas. For each arena, an introductory statement emphasizes key mission themes 
and provides advice. 

Student Progress 

Student progress and achievement come in many forms in community colleges. 
The progress measures suggested here support the assessment of mission effec- 
tiveness by providing an overall picture of student success. They reinforce the 
understanding that student achievement is developmental and that measuring 
it requires tracking both intent and behavior over time. 

The Student Goal Attainment measure seeks to capture individual intent 
and then traces its achievement. It is the most important of the indicators. The 
other two measures, Persistence (Fall to Fall) and Degree Completion Rates, 
follow the behavior of cohorts of students and are less descriptive. They are 
included here in part because they are so frequendy mandated at state and fed- 
eral levels, but also because they can become useful tools when incorporated 
into a larger effort to chart student progress. The danger lies in misinterpreting 
this aggregate data by applying traditional academic assumptions about 
persistence and degree completion to the much broader range of behaviors that 
community college students exhibit. 

Core Indicator 1 : Student Goal Attainment 
Core Indicator 2: Persistence (Fall to Fall) 

Core Indicator 3: Degree Completion Rates 

Workforce Development 

As the job market changes, career programs are not only growing in importance 
but are also constandy responding to shifting expectations. Fewer curriculum 
pathways require exacdy two years to complete, and fewer career ladders relate 
neatly to associate degree preparation. More workers are turning to quick- 
response noncredit programs for career education. They are beginning to 
see initial preparation for work and later upgrading of job skills as part of a 



12 



Core Indicators oj Effectiveness 



continuum; workforce readiness has become an ongoing task that requires a 
variety of programmatic responses. 

Core indicators need to reflect these dynamics and make the assessment of 
various forms of career preparation a routine activity The following guidelines 
will help in this process. First, the evaluation of a student’s career goals should 
not be based on either certificate or degree completion. Objective data such as 
licensure/certification pass rates should be collected, and employers should 
have the chance to provide feedback about their levels of satisfaction. 
Employers need to be able to comment on results relating to student retention, 
promotion, and capacity for career-long growth. Finally, an effort should be 
made to measure noncredit and continuing education functions, as well as 
credit functions in workforce development. 

Core Indicator 4: Placement Rate in the Workforce 

Core Indicator 5: Employer Assessment of Students 

Core Indicator 6: Licensure/Certification Pass Rates 

Core Indicator 7: Client Assessment of Programs and Services 

General Education 

The goal of general education is to strengthen both the general skills and broad 
analytical capabilities that students need in order to function competently in 
day-to-day life. This particular mission arena has grown in importance and com- 
plexity in recent years. Today’s students are expected not only to possess skills 
such as writing and problem solving, but to be able to apply their skills and 
knowledge in an interdependent, culturally diverse world. Community college 
professionals who have long stressed the importance of creating a strong educa- 
tional foundation are now incorporating multicultural themes into across-the- 
curriculum priorities. Equally important, an increasing number of employers 
and community leaders are insisting that college-educated adults perform 
more effectively in the demanding, complex world in which they live. College- 
educated adults must understand diversity and be comfortable with change. 

To assess general education competence, measures in two broad areas are rec- 
ommended: Demonstration of Critical Literacy Skills (communication, critical 
thinking, problem solving, interpersonal skills, etc.) and Demonstration of 
Citizenship Skills (community involvement, multicultural understanding, lead- 
ership, etc.). At a minimum, the evidence gathered in these categories should go 
beyond the accumulation of credits. Students who accrue only a few credits 
should not be included in research, and assessment should focus on feedback 
from those in the community who rely on general education competence. 
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Since general education competencies include both basic and higher-order 
abilities, are developed across the curriculum, and are best demonstrated 
through the application of knowledge, measuring their achievement requires 
sophisticated forms of assessment. Many strategies have been tried with vary- 
ing degrees of success. Community colleges will need to continue to test and 
refine existing methodologies and develop their own local indicators of gener- 
al education competence. Several promising strategies are discussed in the tech- 
nical description in chapter 4. 

Core Indicator 8: Demonstration of Critical Literacy Skills 
Core Indicator 9: Demonstration of Citizenship Skills 

Transfer Preparation 

Although transfer remains an obvious and appealing mission for both internal 
and external audiences, it is a particularly appealing arena for policymakers. It 
is a cost-effective, responsive strategy that broadens the impact of public edu- 
cation by making the first two years of a college education more accessible. 

Unfortunately, too many decision makers continue to work from the 
assumption that community college students will move quickly to transfer 
from high school to community college to baccalaureate attainment. Since 
this is often not the case, assessment of this function must track students in 
order to document transfer patterns. Two indicators are commonly used to 
track transfer students: Number and Rate Who Transfer, and Performance 
after Transfer. Although recording student intent to transfer is a logical and 
essential first step in this process, early statements of student intent are fre- 
quently unreliable. Thus, to avoid mistakes in subsequent measurement, the 
number and rate of actual transfers must be monitored and updated each 
semester or year. 

This line of reasoning suggests the following ground rules: First, request 
information about student intent more than once. Then, follow student behav- 
iors as evidence of intent. Continuous assessment should provide answers to 
the following questions: Have students enrolled for more than one semester, 
earned a designated number of credits, or filed a degree plan? Avoid measuring 
transfer success based on certificate- and degree-completion rates at the com- 
munity college, since many students will follow fluid, open-ended matricula- 
tion patterns. 

Core indicators in this category provide insights at two different levels of 
analysis. The Number and Rate Who Transfer indicator is suited to a sys- 
temwide rather than institutional level of analysis, since transfer goals may vary 
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from community to community and college to college. The Performance after 
Transfer indicator can be usefully compared from college to college. 

Core Indicator 10: Number and Rate Who Transfer 
Core Indicator 11: Performance after Transfer 

Developmental Skills 

By enhancing and reinforcing the skills of underprepared students, develop- 
mental education supports the community college’s ability to achieve its 
mission. Although developmental education has typically been evaluated 
exclusively within the college environment, it is also an area of keen interest to 
policymakers who seek assurances that students receiving remediation perform 
well in subsequent college-level coursework. 

A single core indicator — Success in Subsequent, Related Coursework — is 
proposed under this heading. It presents a useful way for colleges to determine 
how well they are performing as they reach out to an enormously diverse 
student population, many members of which are underprepared. At the same 
time, it provides a reliable measure of return on investment for policymakers 
who are outspoken in their criticism of the public school system and concerned 
about recurring expenditures to teach basic skills. 

In addition to the core measure itself, colleges are encouraged to try to 
collect data for three additional descriptive statistics: the number enrolling who 
need remediation, the number of those who need remediation who are actually 
served, and a follow-up comparison between the achievement of students who 
undertake remediation and those who do not. Although some states currendy 
judge the effectiveness of remediation based on a state-mandated test, a strategy 
that will surely become more common, we propose an indicator that more 
directly assesses classroom performance. 

Core Indicator 12: Success in Subsequent, Related Coursework 

Outreach 

The outreach mission of the community college grows out of the social, 
economic, and cultural circumstances in a local setting. The relationships thus 
formed are unique and interactive. They require that the college remain alert to 
local needs and that it respond with programs and services that reflect its 
institutional capabilities and values. 

The Participation Rate in Service Area indicator describes the involvement 
of community groups and individuals with the college. Such a measure can be 
particularly useful in helping practitioners determine whether there is 
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appropriate, proportional representation from various segments of the commu- 
nity in college programs and services. Because community development 
requires identifying and reaching out to underserved populations in a service 
area, this indicator becomes a valuable mechanism for identifying gaps in service. 

Measuring effectiveness in this arena is not easy. Unlike other aspects of 
mission that are defined in less situational terms, community development 
programs grow out of specific needs, and their evaluation is idiosyncratic. For 
this reason, no one methodology is suggested for the Responsiveness to 
Community Needs indicator, beyond the general rule that colleges must in 
some way identify a client, determine the client’s needs, and measure the client’s 
involvement in and satisfaction with the college’s response. In addition, any 
such measure needs to be monitored over time if it is to become a true index 
of responsiveness. A number of potential methodologies are discussed in 
chapter 4 of this report. 

Core Indicator 13: Participation Rate in Service Area 
Core Indicator 14: Responsiveness to Community Needs 
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Core Indicators: 

A Technical Description 



A s discussed, many of the core indicators do not readily lend themselves 
to measurement. To become operational, they need to be carefully 
defined and carried out. The following pages present technical descrip- 
tions — including a working definition, a description of appropriate data 
sources, and additional measures — for each indicator. 



Student Progress 

Core Indicator 1: 
Student Goal Attainment 



Measure : The proportion of students who, upon leaving community 

college, report that their original goal in attending (or 
subsequent goal decided while enrolled) has been met. 



Data Sources: The recommended method is periodic surveys of the 

student population according to a longitudinal design that 
includes at least the determination of student goal at entry 
• and a follow-up soon after exit to determine if the goal 
has been met or has changed. Most desirable is a 
method whereby student goals are assessed each time the 
student registers. 

Additional An alternative approach to measuring goal attainment is 
Measures: survey items that list particular intended outcomes of 

college on instruments such as entering student question- 
naires, surveys of currently enrolled students, and regular 
follow-ups of former students. Such items typically include 
potential outcomes, such as “learning how to communi- 
cate in writing,” “learning how to work better in groups,” 
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Measure: 



Data Sources: 



Measure: 



Explanation: 



O 




or “learning skills that will enable me to advance on my 
job.” Some 15 to 20 items can be efficiently administered 
using five-point Likert-type scales to determine (a) how 
important the student feels that trait is and (b) how much 
the college has contributed to the student’s development 
of the trait. 



Core Indicator 2: 

Persistence (Fall to Fall) 

Of the cohort of students who register for their first 
credits at a community college in one fall term, the 
proportion that is still enrolled for at least one credit the 
following fall term and has not completed a degree or 
certificate. 

The recommended data source is a continuously updated 
longitudinal database that tracks an identified entering 
student cohort (that is, all students registering for their 
first credits at the community college in a given term) from 
entry to exit using data elements such as course and 
program enrollment and degrees and certificates awarded 
(Ewell, Parker, and Jones 1988; Palmer 1990). 



Core Indicator 3: 

Degree Completion Rates 

The proportion of an entering community college cohort 
officially enrolled in a degree program that actually 
completed a degree or certificate, reported at annual 
intervals. 

The emergence of the federal Student Right- to-Know 
legislation and numerous state reporting require- 
ments has highlighted this statistic as a performance 
indicator, and it is thus impossible to ignore. Recognizing 
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Data Sources: 



Additional 

Measures: 



that community college students take as long as seven or 
eight years to earn a degree, the proposed indicator has no 
end point or time limit. Also, unlike federal and some state 
regulatory requirements, this indicator includes all entering 
students — not just those who are full-time, first-time-in- 
college students. While this may deflate reported graduation 
rates, it more fully reflects actual community college 
populations, the majority of which are in fact part-time 
attenders. Given the sensitivity of this statistic, it should 
never be presented in isolation. It should be reported 
selectively so that attendance patterns, course-taking 
behaviors, and intentions of different student groups are 
apparent. This core indicator should always be reported in 
conjunction with Core Indicator 2, Persistence (Fall to Fall). 

The recommended data source is a continuously updated 
longitudinal database that tracks an entering community 
college student cohort from entry to exit using data elements 
such as date of initial enrollment, degrees or certificates 
awarded, and duration of study. Cohorts should be con- 
structed so that they are consistent with (but more inclusive 
than) those required for current federal and state reporting. 

The following additional statistics are recommended to 
more fully measure effectiveness: 

• reporting data for full-time, first- time-in- college 
students (for example, Right-to-Know students) 
versus other students in the cohort. 

• reporting data according to whether students 
completed 12 credits. 

• reporting data according to whether students reported 
“degree attainment” as an initial or current goal. 

• reporting data for current proportions of students 
attending part-time versus full-time. 

• reporting data for current proportions of students 
reported as successfully transferring or obtaining 
employment without having earned a degree or 
certificate from the community college. 
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Measure: 



Data Sources: 



Additional 

Measures: 



Workforce Development 

Core Indicator 4: 

Placement Rate in the Workforce 

The proportion of an entering community college student 
cohort achieving a “marketable skill” (that is, completing at 
least three occupational or technical courses in a particular 
field of training) that obtains employment in a field direcdy 
related to that skill within one year of last attendance. 

The predominant method used is a follow-up question- 
naire administered periodically to former students by mail. 
If available, state employment information (generally 
available through a state’s Department of Labor or its 
equivalent) provides a more direct method of assessment. 
The obvious limitation of this methodology is the fact that 
many state databases do not contain the job information 
required to adequately identify occupation. 

The following additional measures are recommended to 
better communicate effectiveness: 

• reporting data for students according to the amount 
of prior credit earned at the community college and 
by whether a degree or certificate was earned. 

• reporting data for students according to occupations 
that are both directly and indirecdy related to the 
students’ field of training. 

• a statistic that reports the placement rate of all 
students enrolling for at least one occupational or 
technical course at the community college (this 
relaxes the “marketable skill” restriction in the core 
measure). 
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Core Indicator 5: 
Employer Assessment of Students 



Measure : The proportion of a sample of regional employers in a 

given field who indicate that their employees who received 
training at the community college exhibit skills and job 
performance that are equivalent or superior to those 
exhibited by all employees. 

Data Sources : The recommended method is a periodic cross-sectional 

survey of a sample of area employers using an open-ended 
questionnaire that asks employers to list the specific 
programs and institutions from which they recruit or from 
which they obtain their “best” employees. Because of 
higher response rates, this is preferred to the more 
typical two-phase follow-up method, in which former 
students are first contacted to identify their employers and 
the employers are then surveyed. Multiple attributes of 
performance should be assessed and a Likert-type rating 
scale of at least five points should be used in the 
questionnaire. If resources are available, in-depth 
interviews with employers are recommended. 

To render employer ratings more meaningful, it is 
recommended that they be broken down by field of 
training, by job classification, and by specific performance 
attributes (such as specific job skills, work attitude and 
sense of responsibility, and potential for advancement). 



Additional 

Measures: 



Core Indicator 6: 
LlCENSURE/CERTinCATION PASS RATES 

Measure: The proportion of those who complete or graduate from a 

community college vocational or technical program and 
then actively seek licensure or certification for the first 
time within a given year, who actually obtain licensure or 
certification. 
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Data Sources: The predominant method used is a follow-up questionnaire 

administered periodically by mail to former students or 
those who have completed programs. If available, information 
obtained direcdy from state licensing and certification bodies 
provides a more complete method for compiling such 
statistics. Information obtained in this manner must be 
compiled on a field-by-field basis by contacting the relevant 
licensing or certification body. In some states, lists of such 
bodies are available through the state higher education 
executive office (SHEEO) or the office responsible for 
professional regulation within the state. 

Additional Statistics about overall licensure passage should be 

Measures: broken down by field of study to make them more 

meaningful. A distinction should also be made between 
fields in which licensure or certification is required for 
employment or practice and those in which it is not. To 
reflect the fact that not all fields are governed by such 
recognition, and not all students seek certification in cases 
in which it is voluntary, the proportion of total graduates 
actively seeking certification should also be reported. 



Core Indicator 7: 

Cuent Assessment of Programs and Services 

Measure: No single statistic currendy meets the intent of this indicator. 

Data Sources: Responding to the intent of this indicator requires the 

institution to determine (a) the array of different clients 
that it serves; (b) the nature, types, and volume of programs, 
services, and activities it provides to different clients; and (c) 
the extent to which clients are involved in and satisfied with 
programs, services, and activities. Client describes many 
different individuals, groups, and organizations coming into 
contact with community colleges for many reasons, whose 
involvement and satisfaction must be independently 
determined and assessed. Successful reporting of this 
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Additional 
Measures : 


indicator therefore requires multiple methods for gathering 
information, including surveys, focus group meetings, face-to- 
face and telephone interviews, and participant observation. 

Some of the data sources that can be used to collect 
information consistent with this indicator are the following: 

• specially designed evaluations that are administered 
to the clients who participate in programs and 
services in order to determine their expectations and 
the extent to which the program or service met, 
exceeded, or fell short of expectations. 

• periodic surveys carried out with client groups, typically 
mail or telephone surveys using an instrument designed 
to collect information about both client involvement 
in, and satisfaction with, the program or service. 

° written reports by professional staff members based 
on (a) observation of clients participating in programs 
and services, and their resulting expressions of 
satisfaction, and (b) interviews with clients during 
and after contact with the program or service. 

• focus group meetings of client groups to determine 
their involvement in and satisfaction with programs 
and services. 




General Education 



Core Indicator 8: 

Demonstration of Critical Literacy Skills 



Measure: 


No single statistic satisfies the intent of this indicator. 


Data Sources: 


Although the definition of literacy is shaped by complex 
social forces, this does not absolve community colleges 
from demonstrating that they develop literacy skills. 
Assessing literacy requires a multiplicity of methods 
appropriate to the kinds of literacy tasks that students will 
be facing in the community and on the job. Mere data 
collection is not enough. 
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Once the community college agrees that a particular set of 
literacy skills is an important outcome, it must ensure that 
those skills are (a) taught throughout the curriculum in 
both transfer and occupationally related courses, and (b) 
regularly embodied in local assessments designed by faculty 
Many methods are appropriate to determine literacy skills 
at both the institutional and programmatic levels. Among 
those recommended are the following: 

• faculty-designed instruments to be administered to 
students as appropriate, modeled on such national 
literacy assessments as the National Assessment of 
Educational Progress (NAEP) adult literacy scales. 
Typically, these assessments present students with 
natural literacy tasks such as reading a set of 
instructions, interpreting published charts and 
graphs, completing a required form, or reading and 
interpreting a schedule. 

• specially designed “in-basket” exercises that require 
students to assimilate a variety of materials, interpret 
their significance, and prioritize them for action. 
Exercises of this kind can be embedded in capstone 
experiences that are a part of many applied programs. 

• exercises that require students to read and interpret 
a typical news story. 

• specially designed questions on student follow-up 
surveys that ask about behavior in areas such as cur- 
rent reading habits, perceptions of current events, etc. 



Core Indicator 9: 

Demonstration of Citizenship Skills 

Measure: No single statistic satisfies the intent of this indicator. 

Data Sources: Citizenship is a complex attribute requiring multiple 

evidence-gathering techniques and an institution-wade 
process to ensure that its development and assessment are 
included in instructional delivery. 
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Measure: 



Data Sources: 



The following are among the most promising methods for 
gathering information consistent with this indicator: 

• specially designed questions included in alumni follow- 
up surveys that ask whether the student has voted in 
a local election, has participated in voluntary or com- 
munity service work, or follows the news regularly 

• portfolios documenting citizenship activities that 
students have participated in while enrolled, such as 
volunteer work, memberships in student groups, 
active participation in political campaigns, etc. 

• documentation of student participation in group or 
problem-solving exercises in which leadership and 
group-membership skills, along with appropriate 
occupational or technical skills, can be demonstrated. 

• specially designed exercises included in regular 
course examinations that require students to determine 
the impact of their actions on others or their broader 
responsibilities to others. 



Transfer Preparation 

Core Indicator 10: 

Number and Rate Who Transfer 

The proportion of an identified entering cohort actively 
enrolled in a degree program at the community college and 
completing at least 12 semester hours of credit (or equivalent), 
who then enroll within two years for at least 12 college-level 
credits in a degree program at a four-year institution. 

The preferred data source for this indicator is actual 
student record data obtained from the transfer institution. 
Available methodologies, listed in order of desirability, 
include the following: 

• state-level enrollment information that directly 
matches students who attend community colleges 
and four-year public colleges by means of a student 
identification number. 
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Additional 

Measures: 



Measure: 



Data Sources: 



• electronic transfer of transcript information in a 
defined format. 

• a gg re g ate reporting obtained periodically from 
senior institutions. 

• self-reports obtained from a follow-up questionnaire 
administered to former community college students. 

The following additional measures are recommended to 
better communicate effectiveness: 

• reporting data for students according to the total 
number of credits earned at the community college 
and according to prior degree completion. 

• reporting data for students according to whether 
they explicitly indicated an intent to transfer, 
obtained through a regularly administered survey of 
student intent. 

• reporting the proportion of former community 
college students enrolling for at least one credit at the 
transfer institution (relaxing the 12-hour transfer 
constraint at the senior institution). 



Core Indicator 11: 

Performance after Transfer 

The proportion of regular college-level courses at the 
transfer institution completed with a grade of “C” or better 
by students who previously attended the community 
college, compared with a parallel proportion obtained for 
students who began their studies at the transfer institution 
as first-time freshmen. 

The recommended data source is a continuously updated 
longitudinal database that tracks an entering community 
college student cohort from entry to exit using data elements 
such as date of initial enrollment, degrees or certificates 
awarded, and duration of study. Cohorts should be constructed 
so they are consistent with (but more inclusive than) those 
required for current federal and state reporting. 
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Additional 

Measures: 


This core measure is usefully tabulated for the following 
student groups to better communicate effectiveness: 

• reporting data for students according to the number 
of community college credits earned prior to transfer 
in blocks of 15 credits (for example, fewer than 15 
credits, 15-30, 31-45, 46 or more credits). 

• reporting data for students according to whether 
they earned a degree or certificate before transferring. 




Developmental Skills 



Core Indicator 12: 

Success in Subsequent, Related Coursework 



Measure: 


The proportion of an identified entering student cohort 
that is assessed as deficient in one or more of the basic 
skills (reading, writing, computation), who subsequendy 
(a) successfully completes developmental work intended 
to remediate this deficiency and (b) within one year 
completes their first college-level courses requiring the use 
of this skill with a grade of “C” or better. 


Data Sources: 


The recommended data source is a continuously updated 
longitudinal database that tracks an entering community 
college student cohort from entry to exit using data 
elements such as basic skill proficiency levels, course 
enrollments, and course grades (e.g., Ewell, Parker, and 
Jones 1988; Palmer 1990). 


Additional 

Measures: 


The following additional statistics are recommended to 
further communicate effectiveness on this indicator: 

• the comparative persistence and degree-completion 
rates of students assessed as deficient and remediated 
for each basic skill, compared with similar rates for 
(a) students assessed as proficient and (b) students 
assessed as deficient and not remediated for each 
basic skill. 



A Technical Description 



27 



• the comparative rates of successful college-level 
course completion with a grade of “C” or better for 
these three populations. 



Outreach 



Core Indicator 13: 
Participation Rate in Service Area 



Measure : The proportion of the total population aged 17 or older in 

the college’s service area that has participated in at least 
one organized activity (course, program, service, event, 
etc.) sponsored by the college in the past year. 

Data Sources: The most common source is the college’s records of atten- 

dance at all college-sponsored classes, events, or activities. 
The recommended method is a periodic community 
impact survey administered to a representative sample of 
residents in the service region. This survey should contain 
items describing various types of contact with the 
college (formal and informal), and overall reactions and 
satisfaction rates. 



Additional Unduplicated headcount enrollment and total attendance 
Measures: figures can be usefully reported for this indicator. To get an 

approximate measure of extent of use, another statistic is 
the total number of recorded contact hours (the duration 
of an event times the number of persons participating in 
the event) for all activities (both instructional and non- 
instructional) , divided by the service region population. 



Core Indicator 14: 

Responsiveness to Community Needs 

Measure: No single statistic currendy meets the intent of this indicator. 
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Data Sources: 



Additional 

Measures: 







Responding to the intent of this indicator requires the 
institution to engage in an ongoing process of (a) identifying 
community needs and expectations; (b) demonstrating 
that it is responsive to these needs and expectations by 
continuously improving and adapting its programs and 
services; (c) demonstrating that the constituency groups 
served are satisfied with, and have benefited from, these 
programs and services; and (d) demonstrating that it actively 
seeks ways to establish and promote partnerships with other 
entities and organizations for the benefit of stakeholders or the 
community A community college’s stakeholders will almost 
always consist of distinct constituencies and subgroups whose 
needs and expectations must be independendy determined 
and assessed. Meeting the challenge of this indicator will thus 
require that multiple methods be used for gathering evidence, 
but more important, it will require that an established, 
ongoing process of planning and reviews be used to ensure 
that the evidence is appropriately integrated, interpreted, and 
acted upon by the institution. 

Some of the data sources that can be used to collect information 
consistent with this indicator are the following: 

• periodic needs assessments carried out with citizens 
in the college’s service region; these are typically mail 
or telephone household surveys, using an instrument 
designed to collect information about current 
educational and service needs, and about contact and 
satisfaction with the college. 

• surveys or focus-group meetings with community 
organizations, citizens groups, employers, etc., to 
determine needs and current levels of satisfaction. 

• environmental scanning processes designed to 
systematically examine the content of printed matter 
such as newspaper stories, editorials, job advertise- 
ments, and employment and other socioeconomic 
data about community trends, to determine long- 
term patterns of community development and 
future needs. 
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• specially designed evaluations periodically adminis- 
tered to participants in each program or event 
sponsored by the college, to assess the expectations 
of those participating and the degree to which those 
expectations were met. 

• numbers of partnerships with other organizations or 
agencies in a given year, together with descriptive 
statistics such as the number of individuals served 
through such partnerships, their geographic extent, 
or their total dollar value. 

• internal audits of the institution’s response to a 
selected sample of stakeholder requests — for 
example, to initiate a new program, to offer a new 
service for students or community residents, or to 
engage in a partnership or joint venture. Such audits 
should examine both the time required to respond 
and particular organizational obstacles encountered 
in attempting to do so (see the discussion of 
organizational performance in chapter 5). 
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New Directions 
in Effectiveness 



he core indicators presented in the previous chapter define an appro- 



priate bottom line for demonstrating community college performance. 



They by no means set a limit on what might be measured to improve 
performance. In particular, institutions may want to examine how they func- 
tion as organizations responding to new needs and changing conditions, in 
order to ensure their viability and vitality. It may not be enough, for example, 
to measure student achievement and progress and simultaneously create a mar- 
ket niche. An institutions effectiveness package will need to contain indicators 
that measure not only what students accomplish but also the colleges response 
to what external stakeholders need and expect and how satisfied they are with 
what they get. 

In responding to external stakeholders, an institution should focus on col- 
lecting information about itself to ensure its continuing viability in a changing 
world. For the most part, these data will not be reported to anyone outside the 
college and almost certainly will vary from institution to institution. 

The core indicators omit many items that, for better or worse, institutions of 
higher education are increasingly being asked to measure in the name of 
accountability. Most of these are elements of organizational functioning cen- 
tered on the efficient use of resources. Colleges also need to focus on finding 
ways to deal with inappropriate or poorly constructed measures that outsiders 
may contemplate imposing on the institution. 

New Directions for Assessing Organizational Performance 

If community colleges are to continue as high-performing institutions in the 
future, they will need to address key aspects of organizational functioning. 
Although the following features do not constitute effectiveness in and of 
themselves, they are examples of important dimensions of organizational 
performance for community college leaders to monitor — either as causes of or 
as proxies for quality. 
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1. Ability to accommodate multiple and innovative modes of instructional 
delivery. Responding appropriately to a growing variety of client demands 
requires community colleges to become increasingly flexible in their modes of 
instruction. The days are gone when institutions could simply offer instruction 
on fixed weekly delivery schedules and a standard term-based academic calen- 
dar. At the same time, new modes of instructional delivery that promise to 
increase student learning productivity are becoming more feasible and can be 
widely adopted. Among these are competency-based learning and credentialing 
systems, collaborative learning approaches, service learning approaches, prob- 
lem-based models and simulations, and self-paced or asynchronous learning 
designs (often delivered at a distance through technology). These develop- 
ments will require colleges to develop organizational structures that facilitate 
innovation in instructional delivery. Institutions cannot be burdened with 
administrative procedures that slow their ability to implement innovations. 
Such outdated procedures include time-consuming internal course- and cur- 
riculum-approval processes; faculty reward systems, registration procedures, or 
academic accounting procedures that force course offerings into a term- and 
credit-based format; or simply the attitude that any new way to deliver instruc- 
tion is “special” or exceptional. 

Constructing indicators of an institutions capacity to accommodate new 
modes of instruction might proceed at several levels. At the most basic level, 
the college might monitor the proportion of total instructional activity (mea- 
sured in credit equivalents or enrollment) that is accounted for by nontradi- 
tional scheduling or delivery formats. At a second level, it might examine the 
rate of change: If the proportions are increasing rapidly, there are probably few 
organizational barriers to implementation. Following the logic of Core 
Indicator 14, Responsiveness to Community Needs, a third level might be to 
examine the average time required to gain internal approval or implement new 
instructional approaches, tracking innovations from concrete proposal to the 
point at which the application is up and running. Finally, the institution might 
employ an audit approach and systematically examine the particular organiza- 
tional barriers, incentives, and disincentives that affect a selected set of innova- 
tion initiatives. 

2. Information and planning resources. Rapidly changing environments will 
place increasing burdens on an institutions capacity to generate sound man- 
agement information and to plan for the future. As many corporations have 
found, adequate information capacity is one of the most important ways to 
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ensure vitality in uncertain times. Information and planning resources are 
needed to gain intelligence about trends in the external environment, and this 
knowledge, in turn, may give the institution an advantage in constructing and 
deploying new programs quickly (or in avoiding expensive mistakes). Equally 
necessary are the information and analytical resources required for the institu- 
tion to monitor its own performance and to undertake continuous improve- 
ment. Prominent here are outcomes assessment, measures of stakeholder satis- 
faction, and indicators that examine key internal management and support 
processes. In all these areas, high organizational performance demands more 
than just a stock of data. Information must be widely accessible to staff 
throughout the organization, and it must be prominently and regularly used in 
the decision-making process. 

Constructing indicators for an institution’s information and planning capac- 
ity might also occur at multiple levels. A first step might be simply to examine 
investments in this capacity — items such as administrative computing, data 
acquisition and environmental scanning, and institutional research and plan- 
ning as a percentage of the total operating budget. A good benchmark might 
be 2 percent to 5 percent of total expenditures, the standard commonly applied 
in industry for research and development. 

At the next level, the adequacy of available information and analytical 
resources can be assessed through analysis of the institution’s ability to efficient- 
ly produce useful analytical statistics from its current array of data. Can the insti- 
tution readily calculate such multifaceted performance indicators as the percent- 
age of first-term students taught by part-time instructors, the probability of a 
given first-term student being enrolled in a class with fewer than 20 students, or 
the performance of remediated students in their first subsequent class requiring 
proficiency? A third level might examine information access by documenting 
how and how easily a given member of the community might obtain such data. 

Finally, the actual use of information in decision making might be best deter- 
mined by an audit approach. Here, particular decisions might be selected and exam- 
ined systematically to determine the kinds of information made available and con- 
sidered, or regular internal communications (memos, newsletters, meeting minutes) 
might be scanned to determine if they contain relevant data or references to data. 

3. Asset maintenance and development. The core concept of effectiveness rests 
on an institution’s ability to accomplish valued purposes related to both its 
mission and its stakeholders through the appropriate use of its resources. But a 
purely outcome-based definition of effectiveness may miss the fact that high 
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rates of “production,” in the short run, can be attained by stopping investment 
in key assets. Indeed, this is often what happens in periods of funding shortfall: 
Roofs don’t get patched, equipment doesn’t get replaced, and faculty and staff 
development efforts may cease altogether. Allowing such assets to depreciate is 
rarely good practice in the long run and will ultimately diminish the institution’s 
performance on core outcome measures. Renewing key assets on a regular basis 
effectively positions the institution to accommodate important changes in tech- 
nology and to quickly adopt new instructional or administrative practices that 
may result in significant long-term improvements in effectiveness. 

Constructing indicators that address effective stewardship of key assets is usu- 
ally straightforward. Most such measures are defined as ratios between the cur- 
rent value of an asset and the amount that the institution spends annually to 
maintain or to renew it. Examples include the ratio of annual expenditures on 
facilities repair and renewal versus the current replacement value of the physical 
plant (usually benchmarked at about 2 percent); the ratio of annual expenditures 
on equipment versus the total book value of current equipment (usually bench- 
marked at about 10 percent to 15 percent); the ratio of annual expenditures on 
faculty and staff development versus total personnel compensation; or the per- 
centage of the total course inventory in a given program represented by newly 
developed or redesigned courses (a measure of curriculum renewal). 

Measures like these are never adequate in and of themselves to demonstrate 
institutional effectiveness. But it may be important for an institution to con- 
struct and monitor such measures because they portray organizational factors 
that are demonstrably related to achieving high performance. 

u Red Lights” in Performance Assessment 

At the other end of the spectrum are some “red-light” dimensions of organiza- 
tional functioning that have proved extremely popular with external 
constituencies but that do not appropriately reflect institutional effectiveness as 
we have defined it. Direct measures of efficiency tend to dominate here and reflect 
a desire to shape or control institutional behavior. In discussing such approach- 
es, it is important to stress that effectiveness is about outcomes, not processes. 
The most effective organizations distribute day-to-day management responsibili- 
ties to the lowest feasible level and hold local managers largely responsible for 
outcomes. As long as specified results are achieved on time and within cost, man- 
agers should be given the freedom to deploy institutional resources as they see fit. 
At the very least, when red-light measures are suggested, those proposing them 
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should be reminded that directly mandating institutional behaviors is not likely 
to achieve high performance. This argument will not prevail in difficult political 
circumstances; therefore, it is important to be aware of the specific pitfalls of some 
of the most common red-light measures: 

1. Faculty workload Direct measures of faculty workload are now common for 
many public colleges and universities. This trend reflects a number of external 
concerns. One (less applicable to community colleges than to four-year insti- 
tutions) is the contention that faculty should spend more time on teaching. 
Underlying this sentiment is a general public perception that faculty don’t 
spend much time on any one part of their job. Demands for greater efficiency 
will always lead to the faculty — if only because faculty time is the dominant 
resource at all institutions. 

But measuring faculty workload is a tricky business. A primary concern is 
what to count as work. The most common workload measures tend to look at 
ratios between faculty numbers and teaching outputs like numbers of students 
taught or student credit-hours generated. These are pure productivity mea- 
sures, though, and may or may not reflect real faculty effort. An instructor who 
teaches classes with 50 students looks twice as productive on such measures as 
one who teaches classes of 25, but is the first instructor really working twice as 
hard? Both must design and deliver the same number of class sessions (though, 
admittedly, they have different numbers of assignments to grade). 

Alternative measures tend to look at the number of courses or sections 
taught — an approach that ignores many critical functions that faculty provide, 
such as advising students, designing new programs, or engaging in public 
service. A third common approach tries to measure time allocations direcdy 
through surveys of faculty activity Such surveys are notoriously unreliable 
because individuals may differ in how they report particular kinds of tasks. 

When faced with these three alternatives and forced to make a decision, com- 
munity colleges are probably best advised to adopt the first approach. But in doing 
so, they should make it clear that the resulting statistic can only be interpreted as 
an overall institutional efficiency measure, not as a measure of faculty work. 

Two additional issues are likely to affect discussions of workload: part-time 
instructors and the use of technology Concerns about quality often stimulate 
mandates to measure the proportion of instruction generated by part-time 
faculty This measure may be inappropriate for a community college, because the 
use of part-time professionals is a key ingredient in many occupational or tech- 
nical programs. In other programs, part-timers may be as well or better prepared 
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than their full-time counterparts. Technology also muddies the waters with 
respect to workload. In distance-delivered courses, for instance, a single faculty 
member may reach hundreds of students with the same lecture. At the same time, 
other faculty may be “coaching” students individually through computer-based 
skills-development modules in which students move at their own pace and don’t 
generate traditional credits. Such situations are becoming increasingly common 
and promise to increase substantially in coming years. Capturing them effective- 
ly as part of any traditional faculty workload measure is virtually impossible. 

2. Instructional costs. As emphasized in chapter 2, being cost-effective is a legit- 
imate component of overall effectiveness. This implies that community colleges 
should examine instructional costs as a part of their internal management sys- 
tems. Using cost statistics alone as direct measures of effectiveness for external 
accountability purposes, however, raises numerous issues. 

Because faculty instructional activity is the biggest “factor of production” in 
community college teaching, most instructional cost measures are really variants 
of faculty productivity measures. Outputs are typically counted in the form of 
credits generated or students taught; costs, in turn, are counted in terms of the 
salary dollars allocated to the faculty member teaching each course. The princi- 
pal difficulty here is that faculty salaries vary considerably based on factors that 
have nothing to do with productivity, especially seniority and market factors 
associated with hiring and retaining faculty in particular academic and profes- 
sional fields. Institutions with a particular disciplinary mix — for example, one 
focused on healthcare and technology training — may have naturally higher 
costs than those teaching a different array of fields. The approach of using cost 
statistics also ignores the increasing importance of technology and instructional 
equipment in community college teaching. Directly accounting for all such costs 
in relation to instructional outputs can be a formidable task and will most like- 
ly amplify the effects of differences in disciplinary mix. Indeed, one of the most 
pernicious effects of pure efficiency measures is that they may induce institu- 
tions to stop offering instruction in fields that involve substantial equipment and 
technology investments, such as engineering technology or healthcare. Such 
fields may be precisely the ones most needed for the future. 

3. Degrees granted. The total number of degrees granted by an institution has 
also been prominendy advanced as a measure of community college effective- 
ness. The concern here is the same as that motivating the use of retention-rate 
and completion-rate statistics: that two-year colleges are not producing sufficient 
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numbers of program completers given the numbers of students they enroll. The 
misleading assumption here is that degree production is the only business that 
community colleges should be in. While program completion may be impor- 
tant for some students who enroll, it may be not be a goal of many others. One 
approach to addressing this problem is to survey incoming students to deter- 
mine their actual goals. If degree-granting statistics are mandated for commu- 
nity colleges, information on student goals should also be collected and sup- 
plied. A second way to address the problem is to emphasize statistics related to 
the completion of other goals. With the continuing development of competency- 
based instructional alternatives, certification of achievement or mastery may 
become increasingly common. Reporting the number and proportion of 
students completing such “intermediate” products (more than a course but less 
than a degree) may also help mitigate the negative message for community 
colleges that is always implied by pure degree-production statistics. 

4. Time to degree and “excess” course-taking . A final accountability issue for all 
colleges is the time it takes students to attain a degree. More particularly, 
external authorities are concerned about students taking what appear to be 
more courses than necessary to complete a program, thereby blocking access 
for others. For community colleges, time-to-degree measures like this should 
be particularly resisted. We know that substantial numbers of students enrolled 
in community colleges complete degrees many years after they first enroll. This 
is because the pressures of work and family require them to attend part time or 
to interrupt their enrollment with “stop-out s.” Serving such students is what 
community colleges are, in large part, about. Imposing a full-time student 
model not only ignores the needs of such students but encourages open-access 
institutions to ignore them as well. Part-time attendance and stop-outs are 
largely outside an institution’s control. If excessive time to degree were simply 
a matter of students not being able to find the courses they need, a case for an 
appropriate accountability measure could be made. This is rarely a problem at 
community colleges, as demonstrated by student satisfaction measures like 
those described in Core Indicator 1, Student Goal Attainment. 

Excess-credit measures face similar problems because course-taking is, 
again, largely a matter of student choice. Degree completion is rarely the only 
objective of a student who enrolls in a community college. Many decide on a 
degree only after having completed a number of courses. Others will take 
courses because of particular interests or job-related needs, even though the 
courses are not required for program completion. Finally, available statistics do 
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not suggest that excess course-taking is a particular problem for community 
colleges. On the most commonly used measure — the ratio of total credits actu- 
ally completed by program graduates versus the number required for program 
completion (usually called a “graduation efficiency index”) — two-year colleges 
generally show greater than 90 percent efficiency Statistics on excess course- 
taking should not unduly concern most community colleges. 



Suggestions for Responding to Externally Imposed Measures 

External authorities are becoming increasingly active in imposing direct measures 
of organizational performance on community colleges, in large part because of 
concerns about cost and accountability. As a result, virtually all community col- 
leges will encounter occasions when such measures are advocated. The following 
suggestions may be helpful to a community college faced with this situation: 

1. Lead with the proposed core indicators . The core indicators were construct- 
ed around a model of institutional effectiveness that is appropriate for commu- 
nity colleges and is supported by experience. Based primarily on outcomes, 
they focus on what stakeholders really want from the institutions they invest 
in. Actively leading accountability-related discussions by emphasizing the core 
indicators demonstrates that an institution is willing to be held accountable. At 
the same time, it helps direct attention away from many areas of institutional 
functioning that should be a matter of internal management discretion. 

2. Suggest alternative approaches to examining organizational performance . If 
outcomes are the real bottom line of institutional effectiveness, only those areas 
of organizational functioning that are demonstrably related to outcomes should 
be considered. If external authorities insist on measuring organizational func- 
tioning, they should be reminded that concepts such as organizational flexibili- 
ty and responsiveness, internal information resources and quality assurance 
processes, and the stewardship of key assets are the appropriate areas for a 21st- 
century institution to look at. More important, community colleges should be 
able to demonstrate that they are already looking at these matters internally and 
are making continuous improvements based on what they find. 

3. Actively participate in the process of defining measures that reflect the realities 
of community college clientele and operations. In these days of high accountability, 
living with some kind of performance indicator system is unavoidable. When 
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such measures are proposed, the appropriate response should be to actively work 
with established authorities to make the resulting measures as good as they can 
be. Active resistance to measurement from the outset will only increase pressure 
in return and will most likely result in the eventual imposition of poorly con- 
structed measures. Even the most narrow and inappropriate of proposed mea- 
sures usually can be modified to take into account the particular environment 
within which community colleges operate and the principal clientele they serve. 
Often those who advocate inappropriate measures do not know the service mar- 
ket that community colleges work in and how different it is from that of four-year 
colleges. Faced with persuasive arguments, authorities may be quite willing to 
accept alternatives that meet the basic intent of accountability. 

4. Beware the unaccompanied number No matter how well constructed, statistical 
indicators are always open to misuse and misinterpretation. As a result, colleges 
are well advised to report such statistics together with additional information that 
helps explain why the reported numbers look the way they do. Often this infor- 
mation will consist of facts demonstrating that higher levels of performance have 
actually been achieved. For example, three-year program-completion rates as 
mandated by Student Right-to-Know legislation should always be supplemented 
by program-completion rates at four, five, six, or more years. Such information 
may consist of additional breakdowns or disaggregations of the reported statistics. 
An institution may want to show that its instructional costs appear high in terms 
of total dollars per credit hour because it offers an unusually high proportion of 
technology-intensive programs that its stakeholders want and need. In other cases, 
the accompanying information may consist of important contextual descriptors of 
the population served — emphasizing, for instance, that the colleges student body 
contains high proportions of first-generation, underprepared students or students 
whose primary objective is not to seek a degree. Almost always, the information 
that a community college supplies should be accompanied by statements that 
describe its implications for action, together with what the institution plans to do 
(or is already doing) to address those implications. 

External demands for information are, admittedly, unpredictable and some- 
times unreasonable. Indeed, about the only prediction that can safely be ven- 
tured is that such demands will increase markedly in the years to come. By tak- 
ing a proactive stance in responding to such demands — admitting their legiti- 
macy and helping to move the resulting discussions in a more appropriate 
direction — community colleges can demonstrate their effectiveness and, at 
least in part, avoid being unfairly judged. 
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A Plan for Action 



he changing world of effectiveness evaluation is a mirror of the 



unprecedented change going on in higher education. Market share is 



moving away from the ponderous, the inefficient, and the unaccount- 
able, and toward institutions that focus on and deliver services that meet cus- 
tomer needs. The accountability movement presents an interesting challenge to 
community colleges. In the natural course of operations, it will push colleges 
to accomplish what they are not yet ready to do. It also provides colleges with 
an unparalleled opportunity to clarify their mission for policymakers and to 
better align their programs and services with the needs and expectations of 
their communities. 

Our colleges can respond in many ways. One response is to make effective- 
ness a strategic weapon for demonstrating accountability, that is, to view it as a 
vehicle for achieving success in a turbulent market by documenting perfor- 
mance in ways that meet or exceed stakeholder expectations. If colleges are to 
move in this direction, leaders must act now to demonstrate the effectiveness 
of their institutions. We recommend the following actions: 

1. Collaborate with college stakeholders, both internal and external, to 
determine their needs and what they expect from the college. 

2. Update the college mission statement to reflect stakeholder needs 
and expectations. 

3. State the mission in language that lets the institution assess performance. 

4. Identify a few key performance indicators that demonstrate mission 
attainment and, using these measures, assess institutional 
responsiveness to stakeholder needs. Trying to measure everything 
diffuses an institution’s focus on important priorities. 
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5. Set benchmarks or standards for each performance indicator. 

6. Develop systems for collecting and analyzing performance data. 

7. Use assessment data to enhance decision making and to inform 
stakeholders. 

8. Keep the assessment process simple and flexible. 

9. Be clear in the presentation of data; avoid jargon. 

10. Put performance in context. Submit data on indicators with crisp, 
coherent, factual information focused on institutional mission and clientele. 

1 1. Be proactive; do not wait for quality measures to be imposed on your college. 

12. Establish financial resources and internal management systems to 
support a performance evaluation process. 

The question for community colleges is no longer “Should we be account- 
able? but “How can we best take advantage of the current emphasis 
on accountability and performance to demonstrate the responsiveness and 
effectiveness of our colleges?” 
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